A Computer Vision System for Spaceborne Safety Monitoring

We propose a computer vision system, called a “visual supervisor”, capable of visually monitoring an environment with the goal of safety monitoring and task verification. We demonstrate our system in a spaceborne setting in which from monocular video sequences, it 1) tracks and recognizes objects, 2) detects workspace violations, and 3) supervises in-progress tasks for anomalous situations, failures, or satisfactory progress. The system combines motion segmentation, object tracking, object identification, and pose estimation with cognitive reasoning to construct qualitative interpretations of the scene, such as, “An object, believed to be a satellite, appears to be on a collision course with a second object, believed to be the Space Shuttle.” We demonstrate our framework on the footage of mission STS-87 as viewed from the Space Shuttle Columbia, where the proposed framework correctly identified that the mission had failed.